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SUBJECT:  Report  on  the  Review  of  the  Sensor  Fuzed  Weapon 

as  a  Part  of  the  Audit  of  the  Effectiveness  of  the 
Defense  Acquisition  Board  Review  Process — FY  1992 
(Report  No.  92-050) 


We  are  providing  this  final  report  for  your  information  and 
use.  On  October  24,  1991,  a  draft  of  this  report  was  provided  to 
the  addressees  for  comments.  As  of  February  14,  1992,  we  had  not 
received  responses  to  the  draft  report.  DoD  Directive  7650.3 
requires  that  all  audit  recommendations  be  resolved  promptly. 
Therefore,  all  addressees  must  provide  final  comments  on  the 
unresolved  recommendations  by  April  20,  1992.  See  the  "Status  of 
Recommendations"  section  at  the  end  of  each  finding  for  the 
unresolved  recommendations  and  the  specific  requirements  for  your 
comments.  If  appropriate,  you  may  propose  alternative  methods 
for  accomplishing  desired  improvements.  This  report  does  not 
claim  any  monetary  benefits.  Recommendations  are  subject  to 
resolution  in  accordance  with  DoD  Directive  7650.3  in  the  event 
of  nonconcurrence  or  failure  to  comment. 

The  courtesies  extended  to  the  audit  staff  are  appreciated. 
If  you  have  any  questions  on  this  audit,  please  contact 
Mr.  Russell  A.  Rau  at  (703)  693-0186  (DSN  223-0186)  or 
Mr.  Michael  Welborn  at  (703)  693-0402  (DSN  223-0402).  The 
planned  distribution  of  this  report  is  listed  in  Appendix  F. 


Assistant  Inspector  General 
Enclosure  for  Auditing 

cc: 

Deputy  Secretary  of  Defense 
Secretary  of  the  Air  Force 


Office  of  the  Inspector  General 

AUDIT  REPORT  NO.  92-050  February  18 r  1992 

(Project  No.  1AE-0036.01) 

REVIEW  OF  THE  SENSOR  FUZED  WEAPON  AS  A  PART  OF 
THE  AUDIT  OF  THE  EFFECTIVENESS  OF  THE  DEFENSE 
ACQUISITION  BOARD  REVIEW  PROCESS — FY  1992 

EXECUTIVE  SUMMARY 

Introduction.  The  Air  Force  Sensor  Fuzed  Weapon  was  designed  to 
destroy  multiple  targets  during  a  single  aircraft  delivery 
pass.  The  Sensor  Fuzed  Weapon  must  operate  against  moving  or 
idling  vehicles  during  day  or  at  night  and  in  all  weather 
conditions.  The  Sensor  Fuzed  Weapon  system  is  a  1,000-pound 
class,  unpowered,  unguided,  wide-area  munition  that  consists  of  a 
tactical  munition  dispenser  containing  10  submunitions,  each  with 
4  separate  infrared  sensor  projectiles.  Delivery  aircraft  can 
carry  up  to  four  Sensor  Fuzed  Weapon  munitions. 

Objectives.  Our  overall  objective  was  to  evaluate  the  Defense 
Acquisition  Board  review  process  for  acquisition  of  major  Defense 
acquisition  programs.  In  this  report,  we  evaluated  the  Sensor 
Fuzed  Weapon  program's  compliance  with  applicable  DoD  directives 
and  instructions.  We  also  reviewed  applicable  internal  controls. 

Audit  Results.  The  documentation  that  the  Air  Force  prepared  for 
the  Defense  Acquisition  Board  program  review  in  November  1991 
generally  complied  with  the  requirements  in  DoD  Directive  5000.1 
and  DoD  Instruction  5000.2.  However,  this  review  disclosed 
two  reportable  conditions.  After  the  draft  report  was  issued, 
the  Defense  Acquisition  Board  review  was  rescheduled  for 
March  16,  1992. 

o  Proposed  program-specific  exit  criteria  were  not 
established  for  proceeding  with  future  production  decisions  in 
the  Defense  Acquisition  Board  documentation.  As  a  result, 
management  did  not  have  an  effective  means  of  measuring  progress 
toward,  or  justifying  proceeding  with,  future  production 
decisions  (Finding  A). 

o  Acquisition  program  baselines  did  not  contain  descriptive 
performance  parameters  for  lethality  and  launch  aircraft 
survivability.  Also,  the  baseline  agreement  was  not  approved  by 
the  Under  Secretary  of  Defense  for  Acquisition.  As  a  result,  the 


developmental  baseline  may  not  provide  decisionmakers  sufficient 
insight  into  achievement  of  essential  system  performance 
characteristics  (Finding  B) . 

Internal  Controls.  The  internal  controls  applicable  to  the 
Sensor  Fuzed  Weapon  program  were  deemed  to  be  effective  in  that 
no  material  deficiencies  were  disclosed  by  the  audit.  A 
description  of  the  controls  assessed  is  on  page  2  of  the  report. 

Potential  Benefits  of  Audit.  The  report  did  not  identify  any 
quantifiable  monetary  benefits  (Appendix  D) . 

Summary  of  Recommendations.  We  recommended  the  establishment  of 
specific  exit  criteria  for  future  Sensor  Fuzed  Weapon  production 
decisions  and  acquisition  program  baselines  for  the  number  of 
kills  per  aircraft  pass  required  in  a  countermeasures  environment 
and  the  survivability  of  launch  aircraft  employing  the  Sensor 
Fuzed  Weapon. 

Management  Comments.  No  comments  were  received  in  response  to 
the  draft  report.  Comments  to  the  final  report  are  requested 
from  the  Under  Secretary  of  Defense  for  Acquisition  and  the 
Assistant  Secretary  of  the  Air  Force  (Acquisition)  by  April  20, 
1992. 
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This  report  was  prepared  by  the  Acquisition  Management 
Directorate,  Office  of  the  Assistant  Inspector  General  for 
Auditing,  DoD.  Copies  of  the  report  can  be  obtained  from  the 
Information  Officer,  Audit  Planning  Technical  Support 
Directorate,  (703)  693-0340. 


PART  I  -  INTRODUCTION 


Background 

The  Sensor  Fuzed  Weapon  (SFW)  was  designed  to  destroy  multiple 
targets,  such  as  main  battle  tanks,  armored  personnel  carriers, 
self-propelled  artillery,  trucks,  and  support  equipment,  during  a 
single  aircraft  delivery  pass.  The  SFW  is  intended  to  reduce 
aircraft  attrition  due  to  hostile  fire  during  Air  Force 
battlefield  interdictions.  The  SFW  is  required  to  operate 
against  moving  or  idling  vehicles  during  day  or  night  and  in  all 
weather  conditions. 

The  SFW,  which  is  primarily  intended  to  attack  an  armored 
vehicle's  engine  area,  is  a  1,000-pound  class,  unpowered, 
unguided,  wide-area  munition.  The  SFW  consists  of  a  tactical 
munition  dispenser  containing  10  submunitions.  Each  of  the 
submunitions  contains  4  separate  projectiles,  for  a  total  of 
40  projectiles  in  a  single  SFW  munition.  The  delivery  aircraft, 
including  the  F-15E  and  the  F-16,  can  carry  up  to  four  SFW 
munitions.  The  SFW  is  launched  over  the  target  area,  and  at  a 
predetermined  position,  the  munition  dispenser  releases  the 
10  submunition  delivery  vehicles.  Projectiles,  which  use 
infrared  sensors  to  locate  targets,  are  released  from  the 
submunition  at  a  preset  distance  from  the  ground. 

The  SFW  program  evolved  from  two  separate  Air  Force  weapon  system 
development  programs:  the  wide-area  antiarmor  munition  and  the 
extended  range  antiarmor  munition.  The  feasibility  of  the  SFW 
concept  was  demonstrated  under  extended  range  antiarmor  munition 
validation  contracts  with  Honeywell,  Inc.,  and  Textron  Defense 
Systems  (Textron).  In  1985,  the  Air  Force  selected  Textron  for 
the  SFW  demonstration  and  validation  program  as  a  result  of  a 
competitive  evaluation  of  the  extended  range  antiarmor  munition 
contractors.  In  November  1985,  the  Air  Force  awarded  a  fixed- 
price-incentive-fee  contract  for  SFW  full-scale  engineering 
development  to  Textron. 

Initially,  the  Air  Force  retained  milestone  decision  authority 
for  the  SFW  program  because  the  program  did  not  meet  the  criteria 
for  Defense  Acquisition  Board  (DAB)  cognizance,  as  defined  in  the 
September  1,  1987,  versions  of  DoD  Directive  5000.1,  "Major  and 
Non-Major  Defense  Acquisition  Programs,"  and  DoD  Instruction 
5000.2,  "Defense  Acquisition  Program  Procedures."  Although  the 
SFW  qualified  as  a  major  Defense  acquisition  program  in  1986,  the 
Under  Secretary  of  Defense  for  Acquisition  did  not  formally 
designate  SFW  as  a  DAB  program  until  1988.  The  SFW  program  was 
not  scheduled  for  a  formal  DAB  review  until  September  1991 
(postponed  until  November  1991  and  then  rescheduled  for 
March  1992) . 
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The  Air  Force  plans  to  procure  approximately  16,700-/  SFW  systems 
through  FY  2002.  Total  program  costs  in  then-year  dollars  will 
be  about  $200  million  for  research  and  development  and 
$3.3  billion  for  procurement.  The  acquisition  strategy  provides 
for  three  low-rate  initial  production  buys  leading  to  a  DAB 
Milestone  III  Production  and  Deployment  decision  in 
September  1994.  A  dual  source  competition  is  projected  for  the 
projectile,  and  potentially  the  complete  munition,  in  FY  1996 
with  the  second  source  being  selected  concurrent  with  the  FY  1992 
low-rate  initial  production  buy. 

Objective 

Our  overall  objective  was  to  evaluate  the  DAB  review  process  for 
acquisition  of  major  Defense  acquisition  programs.  In  this 
report,  we  evaluate  the  SFW  program's  compliance  with  applicable 
DoD  directives  and  instructions.  Our  evaluation  of  the 
effectiveness  of  the  DAB  process  and  internal  controls  will  be 
addressed  in  our  summary  report. 

Scope 

To  accomplish  our  objectives,  we  reviewed  documents  that  the  Air 
Force  prepared  and  submitted  to  the  Under  Secretary  of  Defense 
for  Acquisition  as  part  of  the  DAB  review  process  prescribed  in 
DoD  Directive  5000.1,  "Defense  Acquisition,"  and  DoD  Instruction 
5000.2,  "Defense  Acquisition  Management  Policies  and 
Procedures."  We  also  reviewed  data  supporting  the  DAB 
documentation  prepared  by  the  Air  Force  and  the  Defense 
Intelligence  Agency  from  March  1989  to  July  1991.  Appendix  A 
contains  a  list  of  the  DAB  documentation  requested  to  perform 
this  audit.  We  performed  this  audit  from  July  1  through 
October  9,  1991.  Our  audit  was  made  in  accordance  with  auditing 
standards  issued  by  the  Comptroller  General  of  the  United  States 
as  implemented  by  the  Inspector  General,  DoD,  and  accordingly 
included  such  tests  of  internal  controls  as  were  considered 
necessary.  Activities  visited  or  contacted  are  listed  in 
Appendix  E. 


_ /  After  completion  of  audit  field  work,  program  quantities  were 
reduced  with  the  exact  quantity  under  deliberation  at  the 
time  of  report  issuance. 
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Internal  Controls 


As  part  of  the  SFW  program  review,  we  evaluated  the  internal 
controls  associated  with  the  adequacy  of  data  provided  to  the  DAB 
and  deemed  them  to  be  effective  in  that  the  review  did  not 
disclose  any  material  deficiencies  as  defined  by  Public  Law 
97-255,  Office  of  Management  and  Budget  Circular  A-123,  and  DoD 
Directive  5010.38. 

Prior  Audits  and  Other  Reviews 


The  DoD  Inspector  General  and  the  General  Accounting  Office  (GAO) 
have  each  issued  one  report  on  the  SFW  program.  Both  reports  are 
summarized  in  Appendix  B. 

Other  Matters  of  Interest 

On  August  29,  1991,  the  Deputy  Inspector  General  informed  the 
Under  Secretary  of  Defense  for  Acquisition  of  our  findings 
concerning  the  exit  criteria  and  acquisition  program  baselines  on 
the  SFW  program.  Subsequently,  we  discussed  the  subject  of  use 
of  exit  criteria  with  numerous  officials  within  the  Office  of  the 
Under  Secretary.  The  Under  Secretary  of  Defense  for  Acquisition 
issued  a  memorandum  for  DAB  members  on  September  6,  1991, 
concerning  acquisition  program  baselines  (APB)  and  exit 
criteria.  The  memorandum  clarified  key  concepts  regarding  APBs 
and  exit  criteria.  The  Under  Secretary  stated  in  his  memorandum 
that  the  APB  defines  the  overall  acquisition  program  for  a  system 
as  the  user  expects  it  to  ultimately  perform  and  exit  criteria  as 
achievements  for  a  phase  that  measures  progress  toward  meeting 
APB  thresholds.  A  copy  of  the  memorandum  is  in  Appendix  C. 
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PART  II  -  FINDINGS  AND  RECOMMENDATIONS 


A.  EXIT  CRITERIA 

The  Air  Force  did  not  establish  adequate  proposed  program- 

specific  exit  criteria  for  proceeding  with  future  production 
decisions  on  the  SFW  program.  Specifically,  the  Air  Force  did 
not  include  any  proposed  exit  criteria  for  the  FY  1993  and 
FY  1994  low-rate  initial  production  decisions  and  only  general 
exit  criteria  for  the  FY  1995  DAB  Milestone  III  Production  and 
Deployment  decision  in  the  SFW  Integrated  Program  Summary 

document.  Exit  criteria  were  not  required  at  the  time  of  the 

Milestone  II  Full-Scale  Engineering  Development;  therefore,  exit 
criteria  were  not  established  when  the  SFW  became  a  designate  DAB 
program.  Exit  criteria  were  established  to  support  only  the 

first  low-rate  initial  production  decision  by  the  DAB  for  the 
February  1992  program  review.  As  a  result,  DoD  and  Air  Force 
acquisition  management  did  not  have  an  effective  means  of 
measuring  progress  toward,  or  justifying  proceeding  with,  future 
production  decisions. 

DISCUSSION  OF  DETAILS 


Background 

DoD  Directive  5000.1  requires  that  successive  acquisition  phases 
provide  a  systematic,  structured  environment  to  translate  broad 
mission  needs  into  specific  system  requirements.  One  of  the 
primary  means  of  establishing  specific  program  requirements  for 
each  acquisition  phase  is  through  event  driven  acquisition 
strategies  that  explicitly  link  major  contractual  commitments  and 
milestone  decisions  to  demonstrated  accomplishments.  Acquisition 
strategies  should  be  tailored  to  accomplish  established  program- 
specific  objectives  and  to  control  risk.  Acquisition  strategies 
must  also  provide  the  information  essential  for  milestone 
decisions . 

DoD  Instruction  5000.2,  part  11,  section  A  3.e.(2),  defines  exit 
criteria  as: 


.  .  .  the  specific  minimum  requirements 
that  must  be  satisfactorily  demonstrated 
before  an  effort  or  program  can  progress 
further  in  the  current  acquisition  phase 
or  transition  to  the  next  acquisition 
phase.  Failure  to  meet  an  exit  criterion 
halts  the  progress  of  the  system  towards 
the  next  milestone  decision  point. 
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a)  Exit  criteria  are  tied  to  the 
acquisition  phase  in  which  the  program  is 
currently  engaged  and  represent  a  point  on 
the  path  or  growth  curve  toward  the  cost, 
schedule,  and  performance  characteristics 
of  the  system  defined  in  the  acquisition 
program  baseline  for  that  phase. 

b)  Exit  criteria  are  not  always 
performance  parameters,  but  may  be 
training  ev'ents,  test  events,  cost,  or 
contract  provisions. 

Exit  criteria  linked  to  essential  program  accomplishments  provide 
a  means  of  evaluating  a  program's  progress  during  the  current 
acquisition  phase.  Exit  criteria  can  highlight  key  developmental 
events  and  avoid  premature  commitment  to  programs.  In  addition, 
when  exit  criteria  are  properly  incorporated  into  contracts, 
their  use  can  avoid  forcing  program  decisions  solely  because  of 
potential  loss  of  contract  options  that  may  expire  on  a  certain 
date  and  identify  contractor  responsibility  for  the  cost  of 
program  delays  covered  by  events  within  the  contractor's 
control.  DoD  5000. 2M  requires  that  the  Integrated  Program 
Summary  contain  proposed  exit  criteria.  Such  exit  criteria  are 
fundamental  to  acquisition  milestone  or  program  review  decisions 
and  proper  program  oversight  by  DoD  acquisition  officials. 

Potential  Exit  Criteria 


The  March  1992  DAB  review  of  the  SFW  is  being  held  so  that  the 
Under  Secretary  of  Defense  for  Acquisition  can  decide  whether  to 
approve  entering  into  low-rate  initial  production  of  the  SFW, 
formerly  a  Milestone  IIIA  decision  point.  The  Air  Force's 
proposed  acquisition  strategy  recommends  that  production  be 
initiated  in  FY  1992  with  the  first  low-rate  initial  production 
contract  for  98  units  costing  about  $95.7  million.  In  FY  1993, 
the  Air  Force  plans  to  select  a  second  source  for  the  SFW 
projectile  and  award  the  second  low-rate  initial  production 
contract  for  560  units  estimated  at  $252.6  million  to  Textron, 
the  original  supplier.  The  FY  1994  and  FY  1995  production 
contract  awards  will  be  priced  options  on  the  FY  1993  award  to 
Textron.  The  FY  1994  option  is  the  third  low-rate  initial 
production  contract  for  1,364  units  estimated  at  $335.6  million. 
The  FY  1995  option  will  not  be  exercised  until  after  the  Under 
Secretary  of  Defense  for  Acquisition's  approval  at  the  DAB 
Milestone  III  in  late  1994  when  the  SFW  is  scheduled  to  proceed 
into  full-rate  production. 
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Our  review  of  the  DAB  documentation  and  supporting  data  that  the 
SFW  project  office  prepared  in  support  of  the  upcoming  DAB 
program  review  indicated  a  number  of  potential  program,  testing, 
and  development  events  that  could  be  specifically  linked  to 
future  production  decisions  and  form  exit  criteria,  thus 
providing  management  with  measures  of  overall  program  success  in 
terms  of  program  accomplishments.  Exit  criteria  should  also  be 
reflected  in  future  SFW  contracts  in  the  form  of  demonstrated 
events  in  order  to  measure  contractor  performance,  as  called  for 
in  DoD  Instruction  5000.2.  Specifically,  the  Instruction 
requires  that  event  driven  acquisition  strategies  link  program 
decisions  to  demonstrated  accomplishments  in  development  testing 
and  initial  production  and  contract  events  support  the 
appropriate  exit  criteria  for  the  phase  or  intermediate 
development  events  established  in  the  acquisition  strategy. 

For  example,  there  are  contract  events  cited  in  the  first  low- 
rate  initial  production  request  for  proposals  that  were  forwarded 
to  Textron  on  August  5,  1991.  These  events,  which  include  unit 
delivery  and  acceptance  testing,  are  to  be  tracked  by  the 
Government  and  could  be  considered  for  inclusion  as  exit  criteria 
for  future  production  decisions.  Because  additional  contract 
events  may  be  needed  to  adequately  reflect  exit  criteria,  it  may 
be  necessary  to  modify  the  solicitation  to  ensure  DAB  approved 
exit  criteria  are  set  forth  in  contract  events,  as  required. 

Potential  exit  criteria  concerning  the  decision  to  dual  source 
the  SFW  and  the  completion  of  initial  operational  test  and 
evaluation  should  also  be  considered.  For  example,  our  review  of 
the  cost-benefit  analysis  that  the  project  office  prepared  stated 
a  $73  million  savings  to  the  DoD  through  dual  sourcing  the  SFW 
program.  Another  cost-benefit  analysis  performed  by  the  Director 
of  Cost  Analysis,  Deputy  for  Comptroller,  Aeronautical  Systems 
Division,  Air  Force  Systems  Command,  stated  that  DoD  could  lose 
$30  million  if  the  program  is  dual  sourced.  The  results  of  both 
cost-benefit  analyses  were  presented  to  the  OSD  Cost  Analysis  and 
Improvement  Group  and  are  scheduled  to  be  presented  by  the  Group 
as  part  of  the  March  1992  DAB  program  review.  Given  the 
potential  loss  if  the  SFW  is  dual  sourced,  the  continued  cost- 
effectiveness  of  the  dual  sourcing  based  on  the  results  of 
contract  negotiation  and  program  progress  should  be  an  exit 
criteria  for  the  second  and  third  low-rate  initial  production 
decisions  scheduled  for  FYs  1993  and  1994,  respectively. 
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Causes  for  Lack  of  Exit  Criteria 


The  DAB  documentation  did  not  contain  exit  criteria  because  the 
February  1992  DAB  program  review  is  the  first  review  of  SFW  as  a 
DAB  acquisition  program  under  the  February  23,  1991,  5000  series 
of  DoD  acquisition  directives.  Exit  criteria  were  not 
established  for  the  SFW  program  at  the  FY  1986  Milestone  II  Full- 
Scale  Engineering  Development  (now  Engineering  and  Manufacturing 
Development  in  DoD  Instruction  5000.2)  review  because  the 
acquisition  regulations  in  effect  at  that  time  did  not  require 
exit  criteria.  Subsequently,  exit  criteria  were  not  adequately 
established  when  the  SFW  was  designated  a  DAB  program. 

The  Air  Force  did  not  require,  or  previously  establish,  exit 
criteria  in  the  program  except  to  support  entering  low-rate 
initial  production.  The  Air  Force  Program  Director  for  the  SFW 
informed  us  that  in  his  view,  exit  criteria  are  not  required  at 
subsequent  low-rate  initial  production  decision  points,  only  at 
entry  into  low-rate  initial  production.  When  the  exit  criteria 
are  required  will  be  decided  at  the  upcoming  DAB  program 
review.  The  Program  Director  also  stated  that  program-specific 
exit  criteria  would  be  presented  to  the  DAB  for  the  Milestone  III 
decision  at  the  upcoming  program  review.  However,  the  program- 
specific  exit  criteria  were  not  contained  in  the  DAB 
documentation,  which  stated  only  three  general  exit  criteria. 

Effect  of  Mot  Having' Established  Exit  Criteria 

The  absence  of  specific  exit  criteria  geared  to  the  subsequent 
production  decisions  may  impede  OSD's  and  the  Air  Force's  ability 
to  make  subsequent  second  and  third  low-rate  initial  production 
decisions  because  there  are  no  objective  means  of  evaluating 
program  progress  against  the  event  based  acquisition  strategy. 
The  5000  series  of  directives  identifies  a  number  of  requirements 
for  a  Milestone  III  decision,  such  as  Completion  of  a  Beyond  Low- 
Rate  Initial  Production  Report  by  the  Director,  Operational  Test 
and  Evaluation,  which  essentially  are  exit  criteria.  However, 
additional  program-specific  exit  criteria  are  required  based  on 
the  unique  aspects  of  each  program,  including  the  SFW. 

DoD  Instruction  5000.2,  part  11,  section  C,  attachment  1, 
requires  that  the  Under  Secretary  of  Defense  for  Acquisition 
approve  the  exit  criteria  to  be  used  in  the  next  phase  of  a  major 
Defense  acquisition  program  in  the  Acquisition  Decision 
Memorandum.  The  exit  criteria  are  then  intended  to  be  used  to 
ensure  specific  minimum  requirements  are  satisfactorily 
demonstrated  before  the  program  progresses  in  the  current 
acquisition  phase  or  transitions  to  another  phase.  As  a  result, 
the  exit  criteria  should  minimi2e  OSD  program  oversight  during 
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particular  acquisition  phases.  Additionally,  successfully 
completed  exit  criteria  provide  an  objective  and  prompt  means  of 
evaluating  program  progress,  while  failure  to  meet  exit  criteria 
is  an  indication  of  a  need  for  additional  management  attention. 

RECOMMENDATIONS  FOR  CORRECTIVE  ACTION 

1.  We  recommend  that  the  Assistant  Secretary  of  the  Air  Force 
(Acquisition) : 

a.  Direct  the  development  of  proposed  exit  criteria  for 
each  subsequent  production  decision  in  the  Sensor  Fuzed  Weapon 
program,  including  all  low-rate  initial  production  and 
Milestone  III  Production  and  Deployment  decisions. 

b.  Incorporate,  as  appropriate,  the  exit  criteria  approved 
by  the  Under  Secretary  of  Defense  for  Acquisition  as  contract 
events  into  all  Sensor  Fuzed  Weapon  production  contracts. 

2.  We  recommend  that  the  Under  Secretary  of  Defense  for 
Acquisition  establish  program-specific  exit  criteria  for  future 
Sensor  Fuzed  Weapon  production  decisions,  including  low-rate 
initial  production  and  Milestone  III  Production  and  Deployment 
decisions . 


MANAGEMENT  COMMENTS 

We  requested  that  comments  on  the  draft  report  be  provided  to  us 
by  December  23,  1991.  As  of  February  14,  1992,  we  had  not 
received  responses  to  the  draft  report.  Therefore,  we  request 
that  the  Assistant  Secretary  of  the  Air  Force  (Acquisition)  and 
the  Under  Secretary  of  Defense  for  Acquisition  provide  comments 
to  the  final  report. 


STATUS  OF  RECOMMENDATIONS 


Response  Should  Cover : 


Number 

Addressee 

Concur/ 

Nonconcur 

Proposed 

Action 

Completion 

Date 

l.a. 

SAF/AQ 

X 

X 

X 

l.b. 

SAF/AQ 

X 

X 

X 

2. 

USD(A) 

X 

X 

X 
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B.  ADEQUACY  OF  ACQUISITION  PROGRAM  BASELINES 


The  APB  for  the  SPW  program,  while  generally  complete,  did  not 
include  descriptive  performance  parameters  for  SPW  lethality  in  a 
countermeasures  environment  or  for  launch  aircraft 
survivability.  Also,  the  Air  Force  had  not  submitted  the  APB 
agreement  to  the  Under  Secretary  of  Defense  for  Acquisition  and 
the  Joint  Requirements  Oversight  Council  for  review  since 
April  1990  because  the  Air  Force  had  not,  in  its  opinion, 
initiated  revisions  to  the  baselines  that  warranted  further 
approval  until  the  DAB  Milestone  III  Production  and  Deployment 
decision.  The  Air  Force  included  a  lethality  baseline  in  a 
noncountermeasures  environment  that  was  considered  sufficient 
although  it  did  not  reflect  the  SFW's  operational  environment. 
Also,  the  Air  Force  considered  launch  aircraft  survivability  to 
be  implied  in  the  SFW  mission  and  not  required  as  a  separate 
program  baseline.  As  a  result,  the  baselines  by  which  the  SFW 
are  being  developed  and  tested  to  determine  operational 
effectiveness  may  not  provide  decisionmakers  sufficient  insight 
into  achievement  of  essential  system  performance  characteristics 
for  the  SFW  program. 


DISCUSSION  OF  DETAILS 


Background 

To  effectively  control  the  SFW  program,  decisionmakers  in  the 
acquisition  chain  must  be  kept  informed  of  emerging  problems. 
DoD  Instruction  5000.2,  part  11,  section  A,  requires  that  the 
Development  Baseline  include  thresholds  for  the  key  performance 
parameters  using  the  minimum  acceptable  operational  requirements 
identified  in  the  Operational  Requirements  Document  (ORD)  for 
those  parameters.  Before  DAB  reviews,  the  Joint  Requirements 
Oversight  Council  is  responsible  for  validating  these  performance 
goals  and  baselines.  Specifically,  the  Council  validates 
performance  objectives  and  thresholds  proposed  for  acquisition 
program  baselines  of  acquisition  category  I  programs  coming  to 
the  DAB  commencing  at  Milestone  I.  Additionally,  the  Joint 
Requirements  Oversight  Council  reviews  programs  scheduled  for 
milestone  review  to  ensure  that  performance  objectives  and 
thresholds  proposed  for  the  programs  include  a  capability  that 
satisfies  operational  requirements. 

Program  Baseline  for  the  SFW 

Lethality  of  SFW.  The  July  1991  APB  agreement  did  not 
address  the  lethality  required  for  the  SFW  in  an  operational 
environment  with  defensive  countermeasures.  Neither  the 
June  1990  System  Operational  Requirements  Document  nor  the 
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June  1991  Test  and  Evaluation  Master  Plan  addressed  lethality  for 
the  SPW  in  countermeasures  and  noncountermeasures  environments. 
In  our  opinion,  the  baseline  agreement  should  include  the  minimum 
acceptable  lethality  thresholds  and  objectives  stated  in  kills 
per  aircraft  pass  in  both  operational  environments,  as  specified 
in  requirements  documents  and  as  required  in  DoD  Instruction 
5000.2,  part  11,  section  A.  Specifically,  SFW  lethality  as 
measured  by  the  number  of  target  kills  per  aircraft  pass  was 
stated  only  in  an  operational  environment  with  no  defensive 
countermeasures  present.  However,  the  System  Threat  Assessment 
Report  states  that  countermeasures  are  both  likely  and  serious 
threats  to  the  SPW  and  therefore  we  consider  the  countermeasures 
environment  to  be  a  meaningful,  and  perhaps  more  realistic,  basis 
for  an  acquisition  program  baseline.  It  should  be  noted  that 
preliminary  operational  test  and  evaluation  reports  indicated 
that  the  SPW  was  exceeding  performance  parameters  for  lethality 
in  both  required  environments. 

Survivability  thresholds  for  launch  aircraft.  The  Tactical 
Air  Command  did  not  establish  a  launch  aircraft  survivability 
threshold.  In  our  opinion,  this  decision  resulted  in  a 
significant  deficiency  in  the  program  baseline  because  the 
purpose  of  the  SFW  requirement  is  for  multiple  kills  per  aircraft 
pass  and  to  enhance  aircraft  survivability.  Survivability  of 
launch  aircraft,  which  is  a  key  factor  in  delivery  of  ordnance, 
should  be  considered  in  establishing  thresholds  for  performance 
parameters.  If  launch  aircraft  deliver  the  SFW  to  designated 
armored  battlefield  targets  using  tactics  at  altitudes  in  which 
the  SFW  is  considered  most  effective,  the  risk  of  launch  aircraft 
attrition  may  be  greatly  increased. 

According  to  the  SFW  program  office,  the  SFW  will  be  most 
effective  when  the  launch  aircraft  attacks  designated  battlefield 
vehicle  columns  head-on  and  delivers  the  weapon  at  low  altitudes 
over  those  targets.  However,  this  tactic  increases  the  risk  of 
launch  aircraft  attrition  because  the  head-on  attack  also 
increases  launch  aircraft  vulnerability  to  counterattack. 

During  the  March  7,  1991,  SFW  Conventional  Systems  Committee 
planning  meeting,  the  Office  of  the  Assistant  Secretary  of 
Defense  (Program  Analysis  and  Evaluation)  requested  that  the 
Tactical  Air  Command  provide  an  analysis  of  the  SFW's 
effectiveness  in  a  case  of  "no-attrition"  of  launch  aircraft. 
This  analysis  was  to  be  incorporated  as  part  of  the  SFW  Cost  and 
Operational  Effectiveness  Analysis  (COEA).  The  SFW  COEA,  dated 
June  28,.  1991,  stated  that  five  measures  of  system  effectiveness 
were  used  to  assess  the  SFW;  however,  the  Measure  of 
Effectiveness  (MOE)  cost  per  kill  included  all  of  the  four  MOEs 
and  was  the  only  one  that  actually  compared  cost  and  operational 
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effectiveness.  The  four  MOEs  that  made  up  the  cost  per  kill  MOE, 
kills  per  pass,  sorties  per  kill,  munitions  per  kill,  and 
aircraft  attrition  per  kill  were  all  products  of  the  operational 
effectiveness  analysis.  In  turn,  we  found  that  the  COEA 
concluded  that  cost  per  kill  was  most  sensitive  to,  and  driven 
by,  aircraft  attrition.  While  the  COEA  also  concluded  that  the 
cost  benefit  of  SFW  increased  with  higher  aircraft  attrition,  we 
believe  that  an  APB  that  incorporates  aircraft  attrition  and  the 
other  measures  of  effectiveness  (for  example,  cost  per  kill)  is 
required. 

Currently,  kills  per  pass  is  an  established,  APB  parameter  and  is 
the  primary  operational  effectiveness  MOE.  While  kills  per  pass 
directly  influenced  all  other  MOEs  in  the  COEA  including  aircraft 
attrition  per  kill,  aircraft  attrition  does  not  influence  kills 
per  pass.  Therefore,  we  do  not  consider  aircraft  attrition  to  be 
a  part  of  any  of  the  existing  acquisition  program  baselines.  Use 
of  kills  per  pass  as  the  sole  APB  performance  parameter 
associated  with  operational  effectiveness  does  not  provide  for 
consideration  of  aircraft  attrition. 

DoD  Instruction  5000.2  states  that  baselines  should  include  the 
key  cost,  schedule,  and  performance  parameters  established  in  the 
ORD.  DoD  Instruction  5000.2  defines  key  parameters  as  those  that 
if  the  threshold  established  is  not  met,  the  milestone  decision 
authority  would  require  a  reevaluation  of  alternative  concepts  or 
design  approaches.  The  SFW  ORD  does  not  specify  a  performance 
parameter  that  includes  aircraft  attrition,  although  the  ORD 
emphasizes  attack  aircraft  survivability  and  the  ability  of  the 
SFW  to  permit  low  attrition  delivery  profiles  and  multiple  kills 
per  pass.  Specified  performance  parameters  are  related  to 
employment  tactics  that  would  affect  launch  aircraft 
survivability,  although  the  relationship  between  these  tactics 
and  aircraft  survivability  is  not  established  in  the  performance 
parameters.  We  believe  that  this  relationship  should  be 
established,  and  can  be  accomplished,  through  use  of  the  existing 
performance  parameters  and  one  or  more  additional  parameters  that 
specifically  include  aircraft  attrition  in  some  form. 

Under  Secretary  of  Defense  for  Acquisition  approval.  The 
revised  SFW  APB  agreement,  July  14,  1991,  has  not  been  submitted 
to  the  Under  Secretary  of  Defense  for  Acquisition  for  approval 
even  though  the  SFW  is  designated  an  Acquisition  Category  ID 
program.  The  Air  Force  has  indicated  that  there  is  no 
requirement  for  the  Under  Secretary  of  Defense  for  Acquisition, 
who  is  the  milestone  decision  authority  for  the  SFW,  to  approve 
this  document  until  the  Milestone  III  Production  and  Deployment 
decision  because  of  the  SFW  program's  transition  from  an  Air 
Force  to  an  OSD  milestone  decision  authority  after  the 
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Milestone  II  decision  point.  The  Air  Force's  position  is  based 
on  the  Under  Secretary's  approval  of  the  development  baseline  on 
April  17/  1990,  and  the  Air  Force's  determination  that  no  changes 
had  occurred  to  these  approved  baselines.  However,  we  noted  that 
while  the  Air  Force's  conclusion  concerning  the  baselines  was 
correct,  there  were  several  program  changes  highlighted  in  the 
SFW  baseline  agreement  signed  by  the  Assistant  Secretary  of  the 
Air  Force  (Acquisition)  on  July  14,  1991,  that  are  significant 
and  should  be  reviewed  by  the  Under  Secretary  of  Defense  for 
Acquisition.  For  example,  the  President's  FY  1992  budget  reduced 
procurement  funds  for  the  total  program  by  §367.9  million  (then- 
year  dollars)  and  did  not  support  a  previously  approved  Multi- 
Staged  Improvement  Program.  The  Multi-Staged  Improvement  Program 
was  intended  to  improve  overall  operational  effectiveness  by 
increasing  the  number  of  target  engagements.  Also,  the  April 
1990  baselines  did  not  incorporate  additional  baselines  for  the 
SFW  in  a  countermeasures  environment  and  launch  aircraft 
survivability. 

RECOMMENDATIONS  FOR  CORRECTIVE  ACTION 


1.  We  recommend  that  the  Assistant  Secretary  of  the  Air  Force 
(Acquisition) : 

a.  Direct  the  development  of  proposed  performance 
parameters  in  the  acquisition  program  baselines  agreement  for  the 
number  of  kills  per  aircraft  pass  required  in  a  countermeasures 
environment  and  launch  aircraft  survivability. 

b.  Submit  the  Sensor  Fuzed  Weapon  acquisition  program 
baseline  agreement,  including  Air  Force  recommended  baselines,  to 
the  Under  Secretary  of  Defense  for  Acquisition  and  Joint 
Requirements  Oversight  Council  for  review. 

2.  We  recommend  that  the  Under  Secretary  of  Defense  for 
Acquisition  establish  acquisition  program  baselines  for  the 
Sensor  Fuzed  Weapon  program,  including  performance  parameters  for 
lethality  in  a  countermeasures  environment  and  launch  aircraft 
survivability,  after  the  Joint  Requirements  Oversight  Council  has 
concurred  with  the  performance  parameters. 
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MANAGEMENT  COMMENTS 


We  requested  that  comments  on  the  draft  report  be  provided  to  us 
by  December  23,  1991.  As  of  February  14,  1992,  we  had  not 
received  responses  to  the  draft  report.  Therefore,  we  request 
that  the  Assistant  Secretary  of  the  Air  Force  (Acquisition)  and 
the  Under  Secretary  of  Defense  for  Acquisition  provide  comments 
to  the  final  report. 


Number 

l.a. 

1. b. 

2. 


STATUS  OF  RECOMMENDATIONS 


Addressee 


_ Response  Should  Cover ; _ 

Concur/  Proposed  Completion 
Nonconcur  Action  Date 


AF/AQ 

SAF/AQ 

USD(A) 


X  XX 

X  XX 

X  XX 
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PART  III  -  ADDITIONAL  INFORMATION 


Appendix  A 
Appendix  B 
Appendix  C 
Appendix  D 

Appendix  E 
Appendix  P 


-  Documents  Reviewed 

-  Prior  Audits  and  Other  Reviews 

-  Acquisition  Program  Baselines  and  Exit  Criteria 

-  Summary  of  Potential  Benefits  Resulting 
from  Audit 

-  Activities  Visited  or  Contacted 

-  Report  Distribution 


APPENDIX  A:  DOCUMENTS  REVIEWED 


System  Threat  Assessment  Report 
Integrated  Program  Summary 
Development  Test  and  Evaluation  Report 
System  Operational  Requirements  Document 
Program  Life  Cycle  Cost  Estimate 
Independent  Cost  Estimate 
Acquisition  Program  Baseline  Agreement 
Test  and  Evaluation  Master  Plan 
Human  Systems  Integrated  Plan 
Configuration  Plan 

System  Engineering  Management  Plan* 

Functional  Configuration  Audit 
Program  Readiness  Reviews 
Production  Transition  Program 
Results  from  Documentation  Review  by  OSD 
Program  Cost  Estimation 

Initial  Operational  Test  and  Evaluation  Report 
Manpower  Estimate  Report 

Cost  and  Operational  Effectiveness  Analysis 
Acquisition  Plan 

Integrated  Logistics  Support  Plan 
Manufacturing  Plan 
Planning  Meeting  Minutes 
Cost  Analysis  Improvement  Group  Report 
Defense  Acquisition  Executive  Summary  Reports 
Joint  Requirements  Oversight  Council  Assessment 
Defense  Intelligence  Agency  Report 
Cost-Performance  Reports 
Contract  Funds  Status  Report 


*  The  SFW  system  Program  Office  did  not  prepare  a  System 
Engineering  Management  Plan  because  it  is  not  a  requirement  in 
the  full-scale  development  contract.  Further,  the  system  Program 
Office  stated  that  the  information  provided  in  the  System 
Engineering  Management  Plan  is  in  other  documents,  and  therefore 
a  System  Engineering  Management  Plan  would  have  been  redundant 
and  unnecessary. 
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APPENDIX  B:  PRIOR  AUDITS  AND  OTHER  REVIEWS 


General  Accounting  Office  Report  No.  NSIAD-91-235  (OSD  Case 
No.  8718),  "Munitions  Procurement:  Resolve  Questions  Before 
Proceeding  With  Sensor  Fuzed  Weapon  Production,"  August  16,  1991, 
concluded  that  although  the  SFW's  test  results  indicate  that 
technical  problems  have  been  overcome,  the  SFW's  cost  and 
operational  effectiveness  in  its  primary  mission  is  unknown,  and 
threat  changes  could  affect  the  need  for  the  SFW.  GAO  recom¬ 
mended  that  the  Secretary  of  Defense  direct  the  Secretary  of  the 
Air  Force  to  assess  the  SFW's  cost  and  operational  effectiveness 
in  comparison  to  the  full  range  of  interdiction  weapons  using  an 
approved  interdiction  criterion  and  not  approve  the  SFW  for 
production  until  the  Air  Force  conclusively  demonstrates  that  the 
weapon  is  cost-effective  in  its  primary  mission.  DoD  disagreed 
with  GAO's  interpretation  of  the  requirements  for  the  SFW,  which 
DoD  said  led  to  erroneous  conclusions  about  the  criterion  being 
used  to  measure  the  effectiveness  of  the  SFW  for  the  cost  and 
operational  effectiveness  analysis. 

Department  of  Defense,  Inspector  General,  Report  No.  90-072, 
"Acquisition  of  the  Sensor  Fuzed  Weapon,"  May  23,  1990,  concluded 
that  the  SFW  may  not  satisfy  the  mission  requirement  of  the 
Tactical  Air  Forces  because  design  limitations  and  planned 
testing  of  the  SFW  will  not  adequately  assess  required  system 
performance  in  the  postulated  Soviet  threat.  Additionally,  pro¬ 
gram  documentation  and  reports  essential  to  effective  monitoring 
of  the  SFW  full-scale  development  program  were  missing  or  incom¬ 
plete.  The  report  recommended  that  the  Under  Secretary  of 
Defense  for  Acquisition  direct  that  an  independent  Cost  and 
Operational  Effectiveness  Analysis  be  made  to  determine  if  the 
SFW  is  the  most  cost-effective  weapon  system  to  meet  the  inter¬ 
diction  mission  requirements;  the  Under  Secretary  of  Defense  for 
Acquisition  convene  the  DAB  to  decide  on  the  direction  of  the 
SFW;  the  Air  Force  provide  additional  research  and  development 
funds  to  improve  the  SFW;  OSD  revise  DoD  Directive  5000.1  and  DoD 
Instruction  5000.2  to  include  guidance  on  transition  from  minor 
to  major  program  status;  and  the  Air  Force  Systems  Command  submit 
deviation  reports  to  the  Under  Secretary  of  Defense  for 
Acquisition  on  baseline  breaches.  Management's  response 
incorporated  the  known  design  problems  associated  with  SFW 
development,  and  the  Under  Secretary  of  Defense  for  Acquisition 
directed  that  the  Air  Force  report  to  the  Conventional  Systems 
Committee  on  the  status  of  the  program.  As  a  result,  the  Air 
Force  restructured  the  program,  and  the  program  review  was  moved 
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APPENDIX  B:  PRIOR  AUDITS  AND  OTHER  REVIEWS  (cont'd) 


to  September  1991  (now  postponed  until  February  1992).  Because 
of  the  restructure,  additional  research  and  development  funds 
were  required  and  a  Cost  and  Operational  Effectiveness  Analysis 
was  to  be  conducted  before  the  program  review  decision.  The  Cost 
and  Operational  Effectiveness  Analysis  was  to  compare  the  SFW's 
performance  with  competing  antiarmor  weapons  in  meeting  the 
interdiction  mission  requirements  of  the  Tactical  Air  Command. 


22 


APPENDIX  C:  ACQUISITION  PROGRAM  BASELINES  AND 
EXIT  CRITERIA 


THE  UNDER  SECRETARY  OF  DEFENSE 
WASHINGTON.  DC  20301 


6  SEP  1931 


MEMORANDUM  FOR  DEFENSE  ACQUISITION  BOARD  MEMBERS 
SUBJECT:  Acquisition  Program  Baselines  and  Exit  Criteria 


During  the  review  of  the  ATF,  it  became  apparent  to  me  that 
there  is  a  great  deal  of  confusion  over  what  is  contained  in  an 
Acquisition  Program  Baseline  (APB) ,  what  exit  criteria  are  and 
how  exit  criteria  should  be  used,  and  the  relationship  between 
the  two.  The  purpose  of  this  memorandum  is  to  clarify  in  a 
short  space  some  of  the  key  concepts  regarding  APBs  and  exit 
criteria  by  integrating  the  guidance  given  in  Sections  2,  4-B, 
and  11 -A  of  DoD  Instruction  5000.2. 

Attached  is  a  discussion  of  APBs  and  exit  criteria  which 
reflects  the  way  these  concepts  are  defined  and  used  in  DoD 
Instruction  5000.2.  The  attachment  represents  how  I  intend  to 
use  APBs  and  exit  criteria  in  overseeing  acquisition  programs. 


Attachment 


ACQUISITION 
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APPENDIX  C:  ACQUISITION  PROGRAM  BASELINES  AND  EXIT  CRITERIA  (continued) 


ACQUISITION  PROGRAM  BASELINES  AND  EXIT  CRITERIA 

Each  Acquisition  Program  Baseline  (APB)  contains  objectives 
and  minimum  acceptable  requirements  —  known  as  thresholds  --  for 
key  cost,  schedule,  and  performance  parameters.  While  the  level 
of  detail  of  the  APB  evolves  as  the  program  progresses,  DoDI 
5000.2,  Section  11-A  states  that  "values  for  APB  parameters 
reflect  the  cost  and  performance  characteristics  of  the  system  as 
it  is  expected  to  be  produced  and/or  fielded...11  (emphasis  added). 
Exit  criteria,  unlike  the  APB,  are  tailored  to  the  phase  and  are 
described  in  both  Part  2  and  Section  11-A  of  DoDI  5000.2  as 
"program-specific  results  to  be  required  in  the  ...  phase."  Exit 
criteria  are  gates  that  must  be  passed  for  significant  events  to 
occur  during  a  phase,  as  well  as  criteria  which  must  be  satisfied 
at  the  end  of  a  phase  before  passing  to  the  next  phase. 

ACQUISITION  PROGRAM  BASELINES  —  PURPOSE,  CONTENT,  AND  EVOLUTION 

Section  11-A  defines  key  parameters  for  baselines  as  "those 
that  if  the  thresholds  are  not  met  the  milestone  decision 
authority  (MDA)  would  require  a  reevaluation  of  alternative 
concepts  or  design  approaches."  This  means  the  MDA  may  revisit 
the  Milestone  I  or  II  decision  unless  there  is  a  compelling  reason 
to  relax  the  threshold.  Thresholds  and  objectives  in  the  APB 
should  be  determined,  by  and  large,  by  the  interrelated  work  done 
in  the  previous  phase  —  requirements  evolution,  cost  estimates, 
acquisition  strategy  determination,  and  cost  and  operational 
effectiveness  analyses. 

The  identification  of  APB  parameters  is  done  by  both  the 
requirements  validation  authority  and  the  MDA.  The  requirements 
validation  authority  —  the  JROC  for  ACAT  ID  programs  — 
identifies  the  key  operational  performance  (and  schedule,  if 
appropriate)  parameters  in  the  Operational  Requirements  Document 
(ORD),  and  these  parameters  should  be  included  in  the  APB.  x'he 
MDA  may  include  in  the  APB  other  key  performance  parameters 
identified  by  technical  risk  assessment,  cost  and  operational 
effectiveness  analysis,  etc. 

Likewise,  the  specification  of  values  for  the  APB  parameters 
is  done  by  both  the  requirements  validation  authority  and  the  MDA. 
The  values  of  thresholds  for  operational  performance,  and 
occasionally  for  operational  capability  schedules,  are  derived 
from  the  ORD  as  described  in  Section  4-B  of  DoDI  5000.2,  and  ORD 
values  should  be  influenced  by  analyses  as  well  as  military 
judgment.  APB  objectives  for  operational  performance  may  be 
derived  from  the  ORD  but  as  noted  in  Paragraph  2c(3)  of  Section  4- 
B f  they  may  be  influenced  by  other  considerations  such  as  cost  and 
operational  effectiveness  analyses.  Values  for  both  thresholds 
and  objectives  for  non-ope rational  key  parameters  are  specified  by 
the  MDA  based  on  assessments  and  analyses.  Objectives  may  be  the 
same  as  the  threshold  values,  or  they  may  represent  a  meaningful 
increment  beyond  the  threshold  level. 
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APPENDIX  C:  ACQUISITION  PROGRAM  BASELINES  AND  EXIT  CRITERIA  (continued) 


DIFFERENT  PURPOSES r  DIFFERENT  FUNCTIONS 

The  APB  defines  the  overall  acquisition  program  <costf 
scheduler  performance)  for  a  system  as  the  user  expects  it  to 
ultimately  perform  and  the  Department  expects  it  to  cost.  Program 
status  is  measured  and  reported  relative  to  the  APB.  Exit 
criteria  define  program  specific  achievements  for  a  phase  of  the 
acquisition  program  that  are  measures  of  progress  (risk 
reduction),  during  and/or  at  the  end  of  a  phase,  toward  meeting 
APB  thresholds.  Additional  program  activities  or  program  reviews 
are  triggered  by  failure  to  meet  exit  criteria. 


25 


APPENDIX  C:  ACQUISITION  PROGRAM  BASELINES  AND  EXIT  CRITERIA  (continued) 


The  initial  APB  at  Milestone  I,  the  Concept  Baseline,  contains 
a  few  key  cost,  schedule,  and  performance  parameters.  Subsequent 
baselines  (Development  at  Milestone  II  and  Production  at  Milestone 
III)  include  additional,  more  specific,  key  parameters 
representing  the  results  of  tradeoffs  during  the  previous  phase. 
Demonstration  of  these  key  parameters  —  and  other  parameters  in 
the  ORD  —  provide  the  test  data  to  assess  if  the  system  is 
operationally  effective  and  suitable  and  meets  the  mission  need. 

The  higher  order  parameters  in  the  previous  baseline  —  possibly 
refined  as  a  result  of  tradeoffs  and  analyses  in  the  previous 
phase  —  should  be  retained  in  the  new  baseline  unless  no  longer 
judged  to  be  key . 

At  earlier  milestones,  risk  management,  as  described  in  Part 
2,  paragraph  3  of  DoDI  5000,2,  and  the  achievement  of  any  exit 
criteria,  as  described  below,  establish  confidence  in  our  ability 
to  achieve  program  thresholds.  Performance  thresholds  should  be 
demonstrated  prior  to  commitment  to  full  rate  production 
(Milestone  III)  unless  the  particular  parameter  (e.g., 
reliability)  requires  more  test  data  than  can  reasonably  be 
expected  at  Milestone  III.  In  this  case,  a  value  on  a  growth 
curve  should  be  demonstrated. 

EXIT  CRITERIA  —  PURPOSE,  NATURE,  AND  USE 

Durinq  a  phase,  exit  criteria  may  serve  as  "gates"  that,  when 
successfully  passed  (or  exited),  allow  the  program  office  to 
expand  its  activities  or  commitments  within  that  phase  (e.g., 
long-lead  procurement  or  low-rate  initial  production),  with  or 
without  a  formal  program  review.  At  the  end  of  a  phase,  exit 
criteria  are  any  program-specific  accomplishments  required  in 
addition  to  the  minimum  required  accomplishments  for  the  phase 
(listed  in  Part  3  of  DoDI  5000.2)  and  any  other  ADM  direction.  In 
either  case,  exit  criteria  may  be  related  to  performance, 
technology  (e.g.,  demonstrate  a  new  manufacturing  process),  or. 
events  (e.g.,  CDR,  first  flight,  final  assembly).  Exit  criteria 
mr y  be  established  for  a  performance  parameter  in  the  APB  if  the 
demonstration  of  that  performance  parameter  to  some  value  —  not 
necessarily  the  threshold  —  is  critical  to  risk  reduction  for  the 
particular  phase  of  the  program.  This  usage  is  most  likely  for 
the  Dem/Val  phase. 

Exit  criteria  should  be  carefully  and  selectively  applied. 

They  are  intended  to  be  beneficial  to  both  the  MDA  and  the. PM. 

For  the  MDA,  exit  criteria  offer  flexibility  to  set  execution 
boundaries  for  each  phase  of  the  program  and  to  regulate  the 
amount  of  oversight  to  be  applied  during  the  phase.  For  the  PM, 
exit  criteria  offer  the  freedom  to  execute  key  events  during  the 
phase  without  the  formality  of  MDA  and  staff  reviews  except  at 
Milestone  decisions.  However,  if  exit  criteria  are  not  met,  they 
may  delay  progress  or  trigger  a  program  review.  To  be  effective, 
exit  criteria  must  be  specific  and  quantitative.  They  are  not 
intended  to  repeat  or  usurp  the  minimum  required  accomplishments 
for  each  phase  contained  in  DoDI  5000.2,  or  the  APB  objectives  and 
thresholds . 


26 


APPENDIX  D:  SUMMARY  OF  POTENTIAL  BENEFITS  RESULTING  FROM  AUDIT 


Recommendation 

Reference  Description  of  Benefit 

A.l.a.  Compliance  with  Regulation. 

Development  of  proposed 
exit  criteria  for  each 
subsequent  production 
decision  in  the  SFW  pro¬ 
gram,  including  all  low- 
rate  initial  production 
and  Milestone  III 
Production  and  Deployment 
decisions . 


A.l.b. 


A. 2. 


B.l.a. 


Compliance  with  Regulation. 
Incorporate  approved  exit 
criteria  as  contract  events 
into  all  SFW  production 
contracts. 

Compliance  with  Regulation. 
Establish  exit  criteria 
for  future  SFW  production 
decisions,  including  low- 
rate  initial  production 
and  Milestone  III  Production 
and  Deployment  decision. 

Compliance  with  Regulation. 
Development  of  proposed 
performance  parameters  in 
the  APB  agreement  for 
the  number  of  kills  per 
aircraft  pass  required  in 
a  countermeasures 
environment,  and  launch 
aircraft  survivability. 


Type  of  Benefit 
Nonmonetary. 


Nonmonetary. 

Nonmonetary. 


Nonmonetary. 
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APPENDIX  D:  SUMMARY  OF  POTENTIAL  BENEFITS  RESULTING  FROM  AUDIT 
(cont'd) 


Recommendation 

Reference 


B.l.b. 


B.2. 


Description  of  Benefit 

Compliance  with  Regulation. 
Submit  the  SFW  APB  to  the 
Under  Secretary  of  Defense 
for  Acquisition  and  the 
Joint  Requirements 
Oversight  Council  for 
review. 

Compliance  with  Regulation. 
After  the  Joint  Require¬ 
ments  Oversight  Council 
has  concurred  with  the 
performance  parameters 
for  the  SFW,  the  Under 
Secretary  of  Defense 
for  Acquisition  should 
establish  acquisition 
program  baselines  for 
the  SFW  program, 
including  performance 
parameters  for  lethality 
in  a  countermeasures 
environment  and  launch 
aircraft  survivability. 


Type  of  Benefit 
Nonmonetary. 


Nonmonetary. 
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APPENDIX  E: 


ACTIVITIES  VISITED  OR  CONTACTED 


Office  of  the  Secretary  of  Defense 

Office  of  the  Under  Secretary  of  Defense  for  Acquisition, 
Washington,  DC 

Office  of  the  Assistant  Secretary  of  Defense  (Program  Analysis 
and  Evaluation),  Washington,  DC 

Office  of  the  Director,  Defense  Research  and  Engineering 
(Tactical  Warfare  Programs),  Washington,  DC 

Office  of  the  Director,  Acquisition  Policy  and  Program 
Integration,  Washington,  DC 

Department  of  the  Air  Force 

Office  of  the  Assistant  Secretary  of  the  Air  Force  (Acquisition), 
Washington,  DC 

Office  of  the  Assistant  Secretary  of  the  Air  Force  (Financial 
Management  and  Comptroller),  Washington,  DC 

Aeronautical  Systems  Division,  Sensor  Fuzed  Weapon  and  Airfield 
Attack  System  Program  Office,  Eglin  AFB,  FL 

Commander,  Air  Force  Operational  Test  and  Evaluation  Center, 

Eglin  AFB,  FL 

Defense  Agency 

Defense  Intelligence  Agency,  Washington,  DC 
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APPENDIX  F:  REPORT  DISTRIBUTION 


Office  of  the  Secretary  of  Defense 

Deputy  Secretary  of  Defense 

Under  Secretary  of  Defense  for  Acquisition 

Director,  Defense  Research  and  Engineering 

Assistant  Secretary  of  Defense  (Production  and  Logistics) 

Assistant  Secretary  of  Defense  (Program  Analysis  and  Evaluation) 

Comptroller  of  the  Department  of  Defense 

Director,  Operational  Test  and  Evaluation 

Director,  Joint  Staff 

Department  of  the  Air  Force 

Secretary  of  the  Air  Force 

Assistant  Secretary  of  the  Air  Force  (Acquisition) 

Assistant  Secretary  of  the  Air  Force  (Financial  Management  and 
Comptroller) 

Commander,  Aeronautical  Systems  Division 

Program  Director,  Sensor  Fuzed  Weapon  and  Airfield  Attack  System 
Program  Office 

Defense  Agency 

Director,  Defense  Intelligence  Agency 
Non-DoD  Activities 


Office  of  Management  and  Budget 

U.S.  General  Accounting  Office, 

NSIAD  Technical  Information  Center 

Congressional  Committees: 

Senate  Subcommittee  on  Defense,  Committee  on  Appropriations 
Senate  Committee  on  Armed  Services 
Senate  Committee  on  Governmental  Affairs 

Ranking  Minority  Member,  Senate  Committee  on  Armed  Services 
House  Committee  on  Appropriations 

House  Subcommittee  on  Defense,  Committee  on  Appropriations 
Ranking  Minority  Member,  House  Committee  on  Appropriations 
House  Committee  on  Armed  Services 
House  Committee  on  Government  Operations 
House  Subcommittee  on  Legislation  and  National  Security, 
Committee  on  Government  Operations 
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